Skip to content
  • Watch

    Notifications

    Get push notifications on iOS or Android.

Support type classes or implicits #243

Open
baronfel opened this issue on Oct 19, 2016 · 210 comments
Open

Support type classes or implicits #243

baronfel opened this issue on Oct 19, 2016 · 210 comments

Comments

@baronfel

Pick your reaction

Collaborator

@baronfel baronfel commented on Oct 19, 2016

NOTE: Current response by @dsyme is here: #243 (comment)


Submitted by exercitus vir on 4/12/2014 12:00:00 AM
392 votes on UserVoice prior to migration

(Updated the suggestion to "type classes or implicits", and edited it)
Please add support for type classes or implicits. Currently, it's possible to hack type classes into F# using statically resolved type parameters and operators, but it is really ugly and not easily extensible. I'd like to see something similar to an interface declaration:

class Mappable = 
    abstract map : ('a -> 'b) -> 'm<'a> -> 'm<'b>

Existing types could then be made instances of a type classes by writing them as type extensions:

type Seq with
class Mappable with
    member map = Seq.map

type Option with
class Mappable with
    member map = Option.map

I know that the 'class' keyword could be confusing for OO-folks but I could not come up with a better keyword for a type class but since 'class' is not used in F# anyway, this is probably less of a problem.

Original UserVoice Submission
Archived Uservoice Comments

@dsyme dsyme removed the open label on Oct 29, 2016
@dsyme dsyme changed the title Support for type classes or implicits Support type classes or implicits on Oct 29, 2016
@cloudRoutine

Pick your reaction

Collaborator

@cloudRoutine cloudRoutine commented on Oct 29, 2016

For those who haven't seen it, there's an experimental implementation of type classes for F#

Hopefully if this is implemented as a language feature the verbose attribute syntax will be dropped in favor of proper keywords.

It's hard to see how

 [<Trait>]
 type Eq<'A> = 
     abstract equal: 'A -> 'A -> bool 

 [<Witness>] // a.k.a instance
 type EqInt = 
      interface Eq<int> with 
        member equal a b = a = b

presents any advantage over a more terse syntax like

trait Eq<'A> = 
    abstract equal: 'A -> 'A -> bool 

witness EqInt of Eq<int> = 
    member equal a b = a = b

which provides both brevity and clarity

Sorry, something went wrong.

@dsyme

Pick your reaction

Collaborator

@dsyme dsyme commented on Oct 31, 2016

@cloudRoutine There are advantages of a kind.

  • the compiled (and C#-interop) form is considerably more apparent from the first version. You can see that in the above link by the close relationship between existing F# code and the trait-F# version (basically add attributes). It is like looking at the compiled form. This is significant given that Eq probably has good uses as a normal type as well as a class-of-types/trait. It is also significant if you reflect over these, or explicitly instantiate the hidden type parameters
  • it minimizes the actual additions to the F# language design to just a few attributes and a number of special rules. TBH that greatly reduced the number of things that can go wrong when doing the proof-of-concept.
  • adding trait and witness as top-level declarations may be OTT, given that all similar things like Struct and AbstractClass and Literal and so on have been added using attributes. But that can be tuned later.

Note that the prototype has some significant limitations, specifically that if explicit instantiation of the witnesses is allowed, then the "dictionary" being passed can't easily "close" over any values - as envisaged it must be passed by passing Unchecked.defaultof<Dictionary>. Passing actual non-null (and perhaps non-struct-typed) objects as dictionaries might solve this.

Sorry, something went wrong.

@cloudRoutine

Pick your reaction

Collaborator

@cloudRoutine cloudRoutine commented on Nov 3, 2016

It seems I misunderstood how this feature would work. I'd thought that a trait could only be used with witness and that a witness could only use a trait as its interface type.

If an interface with the [<Trait>] attribute can be used like a normal interface, would there be any reason not to adorn every interface with the attribute?

When an attribute needs to be used with a construct in all cases, isn't it effectively the same (from the programmer's perspective) as a top level declaration with extra boilerplate?

I suppose we'll have to rely on tooling to deal with the boilerplate disappointed

Sorry, something went wrong.

@dsyme

Pick your reaction

Collaborator

@dsyme dsyme commented on Nov 3, 2016

If an interface with the [<Trait>] attribute can be used like a normal interface, would there be any reason not to adorn every interface with the attribute?

Off the top of my head there's no reason. We should look at the prototype though (which I felt was in-depth enough to determine questions like this)

Sorry, something went wrong.

@kurtschelfthout

Pick your reaction

Member

@kurtschelfthout kurtschelfthout commented on Nov 10, 2016

Note that the prototype has some significant limitations, specifically that if explicit instantiation of the witnesses is allowed, then the "dictionary" being passed can't easily "close" over any values - as envisaged it must be passed by passing Unchecked.defaultof. Passing actual non-null (and perhaps non-struct-typed) objects as dictionaries might solve this.

I don't understand why this would be useful to allow - assuming you mean something like:

[<Witness>] // a.k.a instance
type EqInt(i:int) = 
    interface Eq<int> with 
        member equal a b = a = b
     member __.TheInt = i

Perhaps I'm missing something but allowing that looks very confusing to me.

Sorry, something went wrong.

@Rickasaurus

Pick your reaction

@Rickasaurus Rickasaurus commented on Nov 10, 2016

I don't mind the attribute style at all. I'm all for keeping the number of keywords in F# as low as possible and building more and more one existing constructs in this manner, avoiding keyword salad.

I do rather like the jargon Rust uses for its type classes (trait and impl) though as I think it's more accessible to normal programmers, witness only makes intuitive sense to people in theorem proving circles, but I'm not super pushing for that to change here just noting my opinion.

One note: because these are written in terms of types it seems like they could never be extended to support statically resolved type parameters. Am I correct?

Sorry, something went wrong.

@Alxandr

Pick your reaction

@Alxandr Alxandr commented on Nov 13, 2016

How would you support stuff like functor?

[<Trait>]
type Functor<'f> =
  abstract fmap: ???

Sorry, something went wrong.

@cloudRoutine

Pick your reaction

Collaborator

@cloudRoutine cloudRoutine commented on Nov 13, 2016

@Rickasaurus can you explain how the example I posted, or one similar to it, creates a "keyword salad"? I don't follow the point you're trying to make. The keyword is already reserved, so it's not like we can use it for ourselves.

because these are written in terms of types it seems like they could never be extended to support statically resolved type parameters. Am I correct?

I can't follow what you mean here either, can you give an example of what you'd like to do?

@Alxandr it can't be supported because we don't have type constructors

Sorry, something went wrong.

@dsyme

Pick your reaction

Collaborator

@dsyme dsyme commented on Nov 14, 2016

@kurtschelfthout

Re explicit witnesses that close over values

I don't understand why this would be useful to allow

e.g. dependency injection (i.e. parameterization of a witness over some dependency):

[<Witness>] // a.k.a instance
type SomeWitness(someDependency: int->int) = 
    interface SomeTypeConstraint<int> with 
        member SomeOperation a b = someDependency a + someDependency b


... SomeConstrainedOperation(SomeWitness(myDependency),...) ...

or


let f () = 
    let myDependency x = x + 1
    ... some declaration that brings SomeWitness(myDependency) into scope ...

   ... SomeConstrainedOperation(...) ... // witness implicitly added here

The utility of this depends on the degree to which you use witnesses to propagate sets of functions which have a non-trivial dependency base. My understanding is that Scala implicits allow this technique. For example, witnesses propagated by implicits may capture a meta programming universe, which is a value.

Sorry, something went wrong.

@Alxandr

Pick your reaction

@Alxandr Alxandr commented on Nov 15, 2016

I still think that not figuring out how to deal with type constructors will severely limit the usefulness of this proposal. Type classes without type constructors would allow for doing abstractions over numerical types, sure, but the lack of ability to do a generic bind and map (and similar) is in my experience what's hurting the most.

Sorry, something went wrong.

@kurtschelfthout

Pick your reaction

Member

@kurtschelfthout kurtschelfthout commented on Nov 15, 2016

@dsyme I see, thanks for the explanation.

I don't have extensive experience with Scala implicits. While allowing bringing witness values into scope is more powerful, you also lose ability to reason about code. Here is the argument in more detail in case anyone is interested: https://www.youtube.com/watch?v=hIZxTQP1ifo

Disallow explicit instantiation of witnesses also means we can make them stateless structs and defaultof always works.

It does mean we would have some shenanigans like having to add wrapper types if one type can be a witness of a trait in more than one way (e.g. 'Monoid' and 'int', via either '+' or '*') but it looks to me like that is the vast minority of cases.

We then also have to think about coherence and orphan instances, e.g if there is more than one possible witness is in scope, warn or error, and have some explicit mechanism to disambiguate. even though this somewhat goes against the current F# practice of resolving names to the first thing in scope. Perhaps it would be enough to disallow orphans (i.e. defining a witness in a compilation unit without also declaring either the trait or the type), which would also cover pretty much all use cases I expect.

@Alxandr What you're asking for are higher-kinded types. The feature request for that is here. I don't think the two should be conflated.

Sorry, something went wrong.

@dsyme

Pick your reaction

Collaborator

@dsyme dsyme commented on Nov 15, 2016

@kurtschelfthout The video is good, thanks

While allowing bringing witness values into scope is more powerful, you also lose ability to reason about code.

I generally prefer arguments in utilitarian terms (bug reduction, safety under refactoring, stability of coding patterns under changing requirements, does a mechanism promote team-cooperation etc.). He makes some of these, though "reasoning about code" is not an end in itself, but can definitely give good utilitarian outcomes. But how many bugs (= lost developer time) are really caused by a lack of coherence, e.g. conflicting instances? I talked about this when last with Odersky and we figured it was very few. But how much time is spent fiddling around with type classes trying to get them to do the right thing, only later hitting some massive limitation like the inability to have local instances, or the inability to abstract instances with respect to a dependency?

A good example where lack of abstraction can hit is string culture. Let's say you create a huge amount of code that uses a swathe of string-related type classes that assume invariant culture (e.g. the ubiquitous show methods). Then you want to localize your code w.r.t. culture, and decide you want to do it properly (rather than via a thread local) and pass the culture as a captured parameter. But your type class instances can't be parameterized. So you either have to remove all those type classes from your code or resort to dynamic argument passing though thread-local state. Painful and discontinuous.

From what I see people in favor of type classes choose examples that are relatively context-free (no local instances required, or only in contrived examples), while people in favour of implicits choose examples where local instances are commonly needed (e.g. Scala meta programming examples, parameterizing by the meta-programming universe implied by a set of libraries). Both sets of examples also put on heavy pressure w.r.t. dependencies (e.g. types-with-nested-types as parameters - the scala meta-programming examples re replete with these) and higher-kinds.

Swift is another important comparison area since it is setting expectations in a new generation of devs.

Sorry, something went wrong.

@drvink

Pick your reaction

@drvink drvink commented on Nov 15, 2016

@dsyme Scala implicits are plagued with problems, the least of which being that Scala's notion of parameterized modules is a runtime phenomenon, leading to issues like being able to have two instances of a Set parameterized by an ordering T and conflate them. This is claimed by some as an intentional benefit, but it seems categorically worse to have this "flexibility" than even the limitations of coherence that are imposed by a naive encoding of Haskell-style type classes, i.e. one lacking more complicated extensions like overlapping instances. The use cases for Scala-style implicits mostly seem to be either bandages for existing code or a form of syntactic sugar at best. Thread-local storage is indeed a hallmark of afterthought-oriented programming, but at least it's fairly explicit and gives no illusions of safety.

@kurtschelfthout Given that it's already possible to encode higher-kinded types to some degree via SRTP, this is probably the best time to have that discussion if we're already having the long-awaited one on type classes for F#, so I don't think @Alxandr is wrong to be bringing it up in this thread. It's difficult to imagine a type class mechanism incapable of Functor/Applicative/Monad bringing significant value; I don't think people want them in F# just so that they can write Show. (CEs are another good example of a feature that would be much more valuable if not for a limitation that feels too extreme; while the clumsiness of composing monads is not specific to F#, CEs and SRTP would at least be complementary features if CE implementation functions--Bind/Return/etc.--were allowed to be static members instead of only members.)

It's worth mentioning that the modular implicits1 proposal for OCaml solves many (all?) of the concerns related to both voiced so far in this thread. There is a shorter and more recent presentation3 from some of the designers as well for those curious.

1: arXiv:1512.01895 [cs.PL]
2: Modular implicits for OCaml - how to assert success

Sorry, something went wrong.

@dsyme

Pick your reaction

Collaborator

@dsyme dsyme commented on Nov 15, 2016

Scala implicits are plagued with problems

Yeah, I know.

... leading to issues like being able to have two instances of a Set parameterized by an ordering T and conflate them. ...

Yes, I know. However TBH I don't think the case is proven this causes bugs in practice. When talking to Odersky about this recently I said that I doubted that a any production bugs had been caused by this problem, or at least vanishingly few. And the workarounds (such as using a unique key type for sets/maps, which is basically what we already do in F# if you need a custom comparer) are not particularly problematic. Certainly the situation is no worse than the existing .NET collections.

Anyway I'd need to see much stronger utilitarian evidence that this truly is as critical as claimed - it seems like a well-intentioned article of mathematical faith (and one which I would wholeheartedly subscribe to from a purely academic perspective) more than one grounded in the reality of software practice. To contrast, the problems of "not being able to parameterize or localize my instances" are very much grounded in modern software reality and standard F# practice. In F#, being able to parameterize a thing by values is very fundamental, even if you have to plumb a parameter to each function of a module or member of a class explicitly. Active patterns, for example, can be parameterized, for good reasons.

The use cases for Scala-style implicits mostly seem to be either bandages for existing code or a form of syntactic sugar at best.

From the F# perspective the whole thing is really syntactic sugar just to pass some parameters implicitly :)

I would like to see an analysis of the extra powers of Scala implicits by someone who favors the mechanism and uses it well, or at least can speak to its use cases. Some of the use cases I've seen in the area of meta-programming look quite reasonable. The mechanism has problems though.

I'll look at the modular implicits work again, it's been a while. Last time I looked it would need extensive modification to be suitable for F#, and it didn't strike me that F# needed to aim for the same goals, but I'll look more carefully. It's a very tricky area to be honest, so many subtleties.

Sorry, something went wrong.

@Rickasaurus

Pick your reaction

@Rickasaurus Rickasaurus commented on Nov 15, 2016

I rather like that Scala will give you an error with an ambiguous instance. Ideally it wouldn't matter, but F# is neither pure nor lazy and so it seems much safer to me to be sure about which instance you're using.

Along these lines I think tooling for this feature might be extremely important. It will certainly be necessary to have an easy way to figure out which instance is being used and where it lives.

@drvink
Side note: I remember suggesting parameterized modules a long time ago, although I wasn't clever enough to see the relationship with type classes back then. What I wanted them for was mostly being able to avoid using classes in cases where some static parameterization was required up front. Also figured it might be used to make certain code more efficient, if the compiler was smart about it.

Modular implicits are pretty neat. I like that they are very explicit with their syntax and so it's more clear to beginners what is going on. One of the weakness (but also paradoxically a great strength) of Haskell is that there is so much magic going on that it ends up taking a lot of mental gymnastics to understand what complex code is doing because so much is inferred. Although, that magic also leads to very terse code.

Sorry, something went wrong.

@kurtschelfthout

Pick your reaction

Member

@kurtschelfthout kurtschelfthout commented on Nov 16, 2016

@dsyme

A good example where lack of abstraction can hit is string culture. Let's say you create a huge amount of code that uses a swathe of string-related type classes that assume invariant culture (e.g. the ubiquitous show methods). Then you want to localize your code w.r.t. culture, and decide you want to do it properly (rather than via a thread local) and pass the culture as a captured parameter. But your type class instances can't be parameterized. So you either have to remove all those type classes from your code or resort to dynamic argument passing though thread-local state. Painful and discontinuous.

You put the many possibilities we already have to propagate values implicitly (statics, thread locals, whatever the thing is called that propagates across async calls) in a negative light, perhaps rightly so. What is the advantage of adding another implicit value propagation mechanism - how is it that much better than the existing ways?

Concretely, in the string culture example. Without implicit value passing, we can change all the witnesses to take CultureInfo.CurrentCulture into account instead of the invariant culture, or refer to some other global. Then we have to make sure that the right value for that is set at the right places in the program. Where this sort of thing needs to be scoped statically I've usually resorted to using blocks in the past, and that seems to work out pretty well.

With implicit value passing, we very similarly have to change all the witnesses to take an extra constructor argument - the culture - and use it in the implementation. And then we have to make sure that the right implicit value is brought in scope at the right places in the program. Perhaps I am missing something but it feels very similar.

On the positive side, my main reasons for supporting this proposal is to:

  1. Support open extensibility - i.e. allow existing types (that I don't control the code for or don't even know exist yet) to be treated as if they implement an interface (trait).
  2. Support what I will loosely call abstraction over constructors - i.e. allow traits with methods like Create : 'T

Don't know if it's me but I keep running into this limitation, and there are no clean workarounds (I know, because I've worked around them many times in different ways). One example is FsCheck allows you to check any type that it can convert to a Property, which is unit, bool, and functions that return any of those (among other things). But the type of check can only be : 'P -> unit note no constraint or indication whatsoever on what this 'P can be, no way for the user to extend allowable types, and consequently hard to document what is actually going on here, leading to much confusion. Something like: Property 'P => 'P -> unit would be so much nicer, esp. if the tooling would catch up and you'd be able to look up straightforwardly what all the witnesses are for Property that are in the current scope. In my estimation, this would significantly reduce the learning curve for new users, improve the documentation, and give advanced users an extra useful (and easily discoverable) extension point.

I realize you can do all of that with implicit values too, because they're strictly more powerful, but I just feel I already have plenty of choice to access values implicitly - perhaps even too many :)

Sorry, something went wrong.

@Alxandr

Pick your reaction

@Alxandr Alxandr commented on Nov 16, 2016

I've used implicits in scala (that being said I've used scala for all of about 2 weeks, so I'm no expert). And what it was used for was passing an execution context around to async code. Basically, it served the purpose of System.Threading.Tasks.TaskScheduler.Current. That being said, implicits might be a better way to handle this than static getters (backed by threadstatic values and other black magic), but I still think that it should be taken into consideration that .NET already has a idiomatic (I think I'm using this word correctly) way of dealing with ambient contexts. And if that needs to be changed I think that's something that should probably be agreed upon by the entirety of the .NET community. I also think these are two different issues. Type classes deals with abstractions of features, whereas implicits are way to implicitly "attach" context to functions. Not to mention the fact that they aren't even mutually exclusive since scala has both (sort of).

Also, I agree with @drvink that while allowing people to implement Show is cool and all, it might also cripple traits into being a niche feature that nobody uses without also figuring out how to deal with type constructors at the same time, with or without CLR support.

Sorry, something went wrong.

@Savelenko

Pick your reaction

@Savelenko Savelenko commented on Nov 18, 2016

@Alxandr As a practicing "enterprise" software engineer, I can assure you, that traits are much needed today, while most of engineers in my immediate environment, which I consider typical, cannot and need not grasp the concept of type constructors in order to be more productive and output better architected programs due to traits. It's just that day-to-day programming does not involve writing (library) code which abstracts over type constructors. But also conceptually, it is not the case that traits as discussed here are "severely limited", because traits/type classes are about polymorphism/overloading, while type constructors are about which terms are considered legal types in a programming language. The two notions are quite orthogonal and we should not mix them here.

Sorry, something went wrong.

@yawaramin

Pick your reaction

@yawaramin yawaramin commented on Nov 20, 2016

Fwiw, my 2c: I've been playing around recently with a very simple dictionary-passing approach to typeclasses (encode the instances as record values holding the operations as functions), see e.g. https://github.com/yawaramin/fsharp-typed-json/blob/ae4c808d3619e3703451211ba2bf079cb6c61bc0/test/TypedJson.Core.Test/to_json_test.fs

The core operation is a function to_json : 'a To_json.t -> 'a -> string which takes a typeclass instance and a value, and converts the value to a JSON string using the typeclass instance. This is fairly simple and easy to implement and use, but the thing that keeps it short of being 'magical' is that I have to manually pass in the instance. Here's the relevant part of the definitions:

module To_json =
  type 'a t = { apply : 'a -> string }
  ...
  module Ops = let to_json t = t.apply

Now, if I could instead mark parameters as implicit, say e.g. with #: let to_json (#t : 'a t) = #t.apply and we had a syntax rule that implicit parameters must always be declared first, perhaps.

And correspondingly declare the instances as: let #string : string t = { apply = sprintf "\"%s\"" }

The compiler would have to convert calls like to_json "a" into to_json #string "a", after finding the implicit with some implicit search mechanism. And that makes it 'magical' again.

Sorry, something went wrong.

@kurtschelfthout

Pick your reaction

Member

@kurtschelfthout kurtschelfthout commented on Nov 24, 2016

Swift is another important comparison area since it is setting expectations in a new generation of devs.

Indeed. I think the closest Swift comes to something like this is through protocols. Compared to interfaces, besides methods and properties they can impose static methods and constructors on the implementing entity. Also some requirements can be specified as optional (you then need to use optional chaining, like the ?. operator in C# to call these. Not really relevant to this discussion). Finally protocols can provide default implementations. So really they are a sort of halfway between interfaces and abstract classes. More possibilities than interfaces, less than abstract classes (in particular they can't define fields), but this allows more flexibility down the line (e.g. a type can be a subtype of multiple traits).

Swift then allows implementing these on types much in the same way as interfaces/abstract classes, but it also allows "protocol extensions". Again comparing to .NET these are like extension methods, but for entire protocols. In this sense, protocol extensions are close to what was proposed in #182.

It's interesting also that like extension methods, protocol extension can impose additional requirements on the extended type at the point of extension using type argument constraints. The example they give is, translated to fictional F# syntax:

//ICollection<'TElement> an existing type
//this extends all ICollections to be also TextRepresentable (another interface/protocol)
//_if_ their elements are also TextRepresentable.
type ICollection<'TElement when 'TElement:TextRepresentable> with
    interface TextRepresentable with
        member self.TextualDescription =
            let itemsAsText = self |> Seq.map (fun elem -> elem.TextualDescription)
            "[" + string.Join(", ", itemsAsText) + "]"

This is very close to how protocols in Clojure work - except they are not typed.

It seems to me that this is qualitatively different from type classes or implicits. In particular, type classes are a static overloading mechanism. Implicits are syntactic sugar to have the compiler pass implicit arguments to functions. UPDATE Protocols allow you to extend dynamic dispatch (the vtable, in some sense) on existing types after the fact. This is wrong, the methods on protocol extensions are statically dispatched, see here and here. I don't know enough about modular implicits in OCaml to comment how it related in one sentence.

In terms of votes this wide range of possibilities for this one suggestion seems problematic, but then of course we have a BDFL @dsyme so the votes are just to appease us unwashed masses anyway ;)

Perhaps it makes more sense to have a goal-directed discussion, instead of focusing on mechanisms. What can't you express right now (or is awkward to express) that you think this suggestion should address? (I gave my 2c on that in an earlier comment)

Sorry, something went wrong.

@dsyme

Pick your reaction

Collaborator

@dsyme dsyme commented on Nov 25, 2016

Perhaps it makes more sense to have a goal-directed discussion,

@kurtschelfthout I'd like to see someone trawl through the various uses of protocols in Swift and pick out 3-4 examples (which couldn't be achieved by OO interfaces, and which feel different in application to type classes)

Sorry, something went wrong.

@kurtschelfthout

Pick your reaction

Member

@kurtschelfthout kurtschelfthout commented on Nov 25, 2016

@dsyme There are a number of use cases of protocols and protocol extensions in the video and slides here: https://developer.apple.com/videos/play/wwdc2015/408/

(note also my update in the comment above - protocol extensions are static constructs, closer to typeclasses than I originally thought, but with more of an OO "feel".).

Sorry, something went wrong.

@dsyme

Pick your reaction

Collaborator

@dsyme dsyme commented 6 days ago

This is the most thumbed-up suggestion in fslang-suggestions and is over 7 years old. Is there any hope this will ever happen?

My position is pretty clear. I'll recap it here.

  1. The utility of type classes for the kind of "functions + data" coding we aim to support in F#, in the context of interoperating with .NET and Javascript, is largely over-rated and has many negative aspects that are rarely considered in a balanced way by those advocating these kinds of features. Some examples of the negative consequences are below:

    • Simple type-classes are never sufficient and result in a consistent stream of requests for more and more type-level programming and type-level abstraction (higher kinds, higher sorted kinds, type families, whatever).
    • Any advanced combination of type-class-like features can very often result in obfuscated, subtle code
    • Adding features in this space leads to a trajectory towards more and more type-level programming. This has serious compilation performance downsides. You end up needing a profiler for your type-level computations, to find out why your compilations are taking so long, or you complain to the compiler implementers about it
    • Adding features in this space leads to a need for compile-time debugging. This is absolutely real - the C++ compiler gives "stack traces" of template instantiation failures, for example. Yet this is a kind of debugging that's completely unsupported in any IDEs today.
    • Adding hierarchical type classification can result in programming communities that spend most their time talking about the "right" way to organise the universe of type classes, and experience surprisingly dogmatic discussions about that
    • Adding hierarchical type classification can result in programming libraries exposed to the "fragile type classification" problem - and repeatedly "rejig" and "tweak" their basic type classifications. This is not possible in a binary compatible ecosystem, meaning we'd be unlikely to ship any hierarchy of classifications in FSharp.Core.
    • Adding type-level programming of any kind can lead to communities where the most empowered programmers are those with deep expertise in certain kinds of highly abstract mathematics (e.g. category theory). Programmers uninterested in this kind of thing are disempowered. I don't want F# to be the kind of language where the most empowered person in the discord chat is the category theorist.
    • The most effective use of these features require end-to-end design of the entire API surface area with these features in mind. Everything becomes tied up into a particular view on type-level abstractions and type-level programming. It quickly becomes impossible to program without an understanding of these abstractions, or without referencing a library which encodes this

    All in all this combination can actually be worse than deeply nested object-oriented hierarchies, for example, which F# also strongly discourages.

  2. We will eventually progress this approved RFC https://github.com/fsharp/fslang-design/blob/main/RFCs/FS-1043-extension-members-for-operators-and-srtp-constraints.md. A pre-requisite to this was https://github.com/fsharp/fslang-design/blob/main/FSharp-5.0/FS-1071-witness-passing-quotations.md and, while that's in F# 5.0, we've still got one outstanding issue related to that. However I am aware that it is an opportunity to resolve all outstanding issues regarding SRTP. I also won't progress it until I am certain that it won't lead to a considerable rise in attempts to use type-level programming in F# for activities outside those designed to be supported by the RFC.

  3. C# has already added features relevant to this space, e.g. static abstract methods. We must consider the ramifications of these for F# and integrate these.

  4. We will not progress a type class design independently of C# (beyond SRTP, FS-1043, static abstract methods and so in)

  5. TypeScript and typed Python are incorporating type-level programming programming features that are practically based, for interop/API-typing purposes. These features have many of the problems described above, but are at least using a methodology far more relevant to F# than the "hey, let's make programming maximally abstract, maximally inferred and as much like category theory as possible" agenda sometimes pursued. That is, if we add more type-level programming features they would be much more likely to be the sort of thing to support the interop needs of Fable or F#-interop-to-Python or similar.

As an aside, something strange happens when one tries to have rational conversations about the above downsides with people strongly advocating expansion of type-level programming capabilities - it's almost like they don't believe the downsides are real (for example they might argue the downsides are "the choice of the programmer" or "something we should solve" - no, they are intrinsic). It's like they are prioritising type-level programming (computation that happens at compile-time) over actual programming (computation that happens at runtime). Indeed I believe psychologically this is what is happening - people believe that programming in the pure and beautiful world of types is more important than programming in the actual world of data and information. In contrast, the downsides are real and can have a thoroughly corrosive effect on the simple use of a language in practice in teams.

As one example, I've translated Swift code using protocols to F#, and the use of Swift protocols was needless, and the resulting F# was much, much simpler (and much more like the original Python). It was very obvious to me that the person translating the original Python samples used Swift protocols to "show off" what so-called "value" Swift was bringing to the table. Well they failed - the use of protocols was simply obfuscating, not helpful. This happens all the time once extensive type-level programming facilities are available in a language - they immediately, routinely get mis-applied in ways that makes code harder to understand, excludes beginners and fails to convince those from the outside.

Sorry, something went wrong.

@Happypig375

Pick your reaction

Contributor

@Happypig375 Happypig375 commented 6 days ago

What happened? Why delete? Why edit a large comment to "."?

(Edit for context: 2 deletions happened before the previous comment, and the one of the edit histories of the previous comment was just a .)

Sorry, something went wrong.

@dsyme

Pick your reaction

Collaborator

@dsyme dsyme commented 6 days ago

What happened? Why delete? Why edit a large comment to "."?

My apologies :)) I got in a huff and deleted two messages that were perfectly reasonable - I don't even know why I did it - there was absolutely nothing wrong with the messages, the whole thread was just getting a bit off track. Anyway the messages were totally fine and I will find the original text of the messages and repost them below.

Comment #1 By robertj

Why catch up? If it all C# will have them first (if at all) and only then F# will implement such a feature

Comment #2 By SchlenkR

IMHO the way F# progresses seems to be „fear-driven“ since a while. The fear is that scenarios like async/task, quotations/linq, tuples, etc. will occur again when F# is evolving in major steps, so it’s evolving in minor steps only. Although I personally would like to see F# progression going another way, I can understand the decisions being made and they are legitim, even if I don’t share them. There are still heavily weighting argument to choose F# in favour of C#, and these reasons are fundamental to me (HM type inference, expression based, some more), and C# will never „catch up“ with those I guess. And if I had the choice of having TC/HKT, and „F# having the best IDE support of all languages“, it would be a really hard decision. What I want to say is: there are many „minor“ things that can be done to improve the user experience of F# on a major level, and I would be happy to see progression in those fields, too. And one day... I will find a customer using Haskell only blush

Again, my apologies

Sorry, something went wrong.

@dsyme

Pick your reaction

Collaborator

@dsyme dsyme commented 6 days ago

I have added a link to my response within the text of the main suggestion.

Sorry, something went wrong.

@JustinWick

Pick your reaction

@JustinWick JustinWick commented 6 days ago

What happened? Why delete? Why edit a large comment to "."?

My apologies :)) I got in a huff and deleted two messages that were perfectly reasonable - I don't even know why I did it - there was absolutely nothing wrong with the messages, the whole thread was just getting a bit off track. Anyway the messages were totally fine and I will find the original text of the messages and repost them below.

It's my hope some day to write something truly great here that's just edgy enough to get deleted, but reasonable enough that that person feels bad about deleting it.

That's the F# I want to see in the world--something beautiful that's just a little too much.

(Note: this comment is one of those defuse-a-situation jokes and should not be taken seriously)

Sorry, something went wrong.

@matthewcrews

Pick your reaction

@matthewcrews matthewcrews commented 6 days ago

As someone who believes they would greatly benefit from Type Classes, I would like to offer a thought. There is no idea that I have NOT been able to express in F# as it exists today. Now, I may have to write more method overloads than I would like, but it is expressible. The lack of Type Classes has pushed my design in different directions than I wanted at first but I was still able to achieve the desired outcome.

Sorry, something went wrong.

@drvink

Pick your reaction

@drvink drvink commented 6 days ago

Since this topic is so frequently revisited, it's worth mentioning that Haskell-style typeclasses can be fully encoded in F# today, and those who wish to use some of the the "famous" abstractions (Functor, Applicative, Monad, etc.) can have them off-the-shelf via F#+; furthermore, for those interested in the actual encoding via F#'s constraint mechanisms, @cannorin has written a lovely article in Japanese which happens to machine translate quite well for anyone who wants to understand the ideas and/or extend them for their own purposes.

I can't help but mention as a side note (as an also-Haskell user!) that the experience of writing out explicit constraint invocations in F# (and burying them in implementation details/a library) and with F#+ has been less painful than extensive use of type classes in GHC/Haskell itself. Make of that what you will (though I did pay my dues many years ago for both.)

Sorry, something went wrong.

@DalekBaldwin

Pick your reaction

@DalekBaldwin DalekBaldwin commented 6 days ago

"C# has already added features relevant to this space, e.g. static abstract methods. We must consider the ramifications of these for F# and integrate these."

Is there an issue for tracking this yet? It's already a pleasure to use in C# preview.

(Note that this feature, on its own, allows definitions of typeclasses of kind * like Semigroup/Monoid.)

Sorry, something went wrong.

@vzarytovskii

Pick your reaction

@vzarytovskii vzarytovskii commented 6 days ago

"C# has already added features relevant to this space, e.g. static abstract methods. We must consider the ramifications of these for F# and integrate these."

Is there an issue for tracking this yet? It's already a pleasure to use in C# preview.

I haven't seen one yet. If you can create feature request to track it, it would be great!

Sorry, something went wrong.

@kspeakman

Pick your reaction

@kspeakman kspeakman commented 6 days ago

I also won't progress it until I am certain that it won't lead to a considerable rise in attempts to use type-level programming in F# for activities outside those designed to be supported by the RFC.

I agree with this. Just please don't go full Elm lockdown. laughing rofl

(Not that I want TCs. I pretty much only use the basic features and get along fine.)

Sorry, something went wrong.

@2A5F

Pick your reaction

@2A5F 2A5F commented 5 days ago

In fact, people need hkt and associated type more than type classes

Sorry, something went wrong.

@ShalokShalom

Pick your reaction

@ShalokShalom ShalokShalom commented 5 days ago

And HKT are confirmed in principle, we just wait for C# on this one as well.

#175

Sorry, something went wrong.

@kspeakman

Pick your reaction

@kspeakman kspeakman commented 4 days ago

@ShalokShalom Do you know where dsyme has commented on this as approved? I didn't find it in the RFCs. The archived user voice link on the issue is broken. Does the "needs clr change" tag imply approval? I did not take it that way.

Sorry, something went wrong.

@TheInnerLight

Pick your reaction

@TheInnerLight TheInnerLight commented 2 days ago

The utility of type classes for the kind of "functions + data" coding we aim to support in F#, in the context of interoperating with .NET and Javascript, is largely over-rated and has many negative aspects that are rarely considered in a balanced way by those advocating these kinds of features.

I think if you were to visit a variety of language communities, you would find it a prevalent view that that language has somehow hit upon an excellent set of features that has the perfect balance of cutting edge techniques and pragmatism. In reality, that's probably not the case, it is considerably more likely that those communities have learnt about some of the available cutting edge techniques, are familiar with how much easier those techniques make their life and are willing to advocate for them but are not willing to do the same for other techniques which are, to them, unproven and unfamiliar.

To give an example where people here probably have no skin in the game, Haskell has terrible out of the box support for Strings, treating them as a [Char]. It's slow, it uses huge amounts of memory and it's a barrier to entry because you have to learn not to use something that it seems like you obviously should use.

Haskell also lacks support for String interpolation. There are those in the Haskell community who would argue that the status quo is totally fine, you can pull in the text package after all and whatever other library support you want to use for better String interpolation. They might argue that fixing the problem has negative aspects that are rarely considered by proponents, for example, tying more code to GHC release cycles. Meanwhile, in the world of .NET, both C# and F# have made considerable strides in this area, releasing features that make people's lives easier when working with Strings.

To people inside the Haskell community, it's easy to workaround this issue and barely registers because doing so is so commonplace. To people outside the Haskell community, it's an immediate example of Haskell's perceived impracticality for production use-cases.

In short, it is easy and natural to overvalue the status quo, we should be aware that this is our likely bias and try to consider how decisions might look to those outside a community who might wish to join as well as those on the inside.

  • Simple type-classes are never sufficient and result in a consistent stream of requests for more and more type-level programming and type-level abstraction (higher kinds, higher sorted kinds, type families, whatever).

This is a classic slippery slope argument and it's worth considering a number of related questions:

  • Is the idea of simple type classes a worthy idea in its own right?
  • Is the inclusion of simple type classes likely to result in such a clamour for new features like higher kinds, higher sorted kinds, type families that they cannot each be considered objectively in their own right?
    • Has any general purpose language implemented higher sorted kinds or is this just reaching for the absurd?
  • Is the idea that people might ask for other language features so terrible that it deserves excluding useful techniques on that basis?
  • Any advanced combination of type-class-like features can very often result in obfuscated, subtle code

Obfuscation is something done intentionally. As for subtlety, I wonder if you can provide an example of what you're thinking of? It seems that there are plenty of things in F# that can already produce subtle code in much more egregious ways, the combination of side effects and lazy evaluation springs to mind.

Are the subtleties introduced by type classes notably worse than the subtleties introduced by any other language feature that has been successfully integrated with the language?

  • Adding features in this space leads to a trajectory towards more and more type-level programming. This has serious compilation performance downsides.

On the plus side, more type-level programming has massive run-time performance upsides. This has been notable in the Scala world as the transition from reflection-based JSON codecs to typeclass-based JSON codecs resulted in dramatically faster code as well as fewer weird runtime errors.

  • Adding features in this space leads to a need for compile-time debugging. This is absolutely real - the C++ compiler gives "stack traces" of template instantiation failures, for example. Yet this is a kind of debugging that's completely unsupported in any IDEs today.

The Haskell and Scala compilers have managed to avoid this but Scala IDE tooling is able to provide information on implicit resolution.

  • Adding hierarchical type classification can result in programming communities that spend most their time talking about the "right" way to organise the universe of type classes, and experience surprisingly dogmatic discussions about that

Programming communities tend to spend their time talking about lots of surprising things. Surely it's positive that you have an active community that is talking about things rather than a disengaged, uninterested community?

Why do you think that these conversations are less useful or productive than any of the other topics that come up in language communities like the millionth iteration of the best form of dependency injection?

  • Adding hierarchical type classification can result in programming libraries exposed to the "fragile type classification" problem - and repeatedly "rejig" and "tweak" their basic type classifications. This is not possible in a binary compatible ecosystem, meaning we'd be unlikely to ship any hierarchy of classifications in FSharp.Core.

I'm not quite sure what the "fragile type classification" problem is because it doesn't appear to have ever been mentioned outside of this thread.

I'm going to guess that you're talking about hierarchical relationships between type classes such as the Functor, Applicative, Monad relationship. Such things change quite rarely in practice but wouldn't necessarily need to be provided by FSharp.Core in any case. In Scala, these are provided by an external library, Cats.

  • Adding type-level programming of any kind can lead to communities where the most empowered programmers are those with deep expertise in certain kinds of highly abstract mathematics (e.g. category theory). Programmers uninterested in this kind of thing are disempowered. I don't want F# to be the kind of language where the most empowered person in the discord chat is the category theorist.

Adding any new feature results in those who know how to use that feature having an advantage over those who don't yet know how to use it. This is great because it's an opportunity for people to learn things. If we didn't like learning things, we'd never have learnt F#.

Indeed, very easily, at the beginning of F#'s days, you could have said something like: most programmers don't know how to use map, let's not put that in the language, it will just lead to Haskell and ML engineers knowing everything and everyone else feeling disempowered.

Since F# was created, far more engineers worldwide have become comfortable with functions like map, filter and fold and lots of languages have added these concepts. F# was one of a set of languages that helped spread that understanding - win!

The question is, do you want F# to be setting the agenda like it used to or do you want to be Go, currently adding the cutting edge language features of the early 2000s, like Generics that no modern language should've been built without?

  • The most effective use of these features require end-to-end design of the entire API surface area with these features in mind. Everything becomes tied up into a particular view on type-level abstractions and type-level programming. It quickly becomes impossible to program without an understanding of these abstractions, or without referencing a library which encodes this

Fortunately, type-classes, unlike interfaces, don't require access to the underlying type to implement. Cats in the Scala-world is able to implement lots of Haskell-inspired abstractions without requiring the language to commit to them.

Scala engineers are thus empowered with choice as to whether to write code just with the standard library tools or whether to adopt Haskell-like abstractions and use Cats/Cats Effect and various other libraries that facilitate purely functional Scala.

As an aside, something strange happens when one tries to have rational conversations about the above downsides with people strongly advocating expansion of type-level programming capabilities - it's almost like they don't believe the downsides are real (for example they might argue the downsides are "the choice of the programmer" or "something we should solve" - no, they are intrinsic). It's like they are prioritising type-level programming over actual programming. Indeed I believe psychologically this is what is happening - people believe that programming in the pure and beautiful world of types is more important than programming in the actual world of data and information. In contrast, the downsides are real and can have a thoroughly corrosive effect on the simple use of a language in practice in teams.

Person A: "No programmer obsesses over type-level programming."
Person B: "But my uncle Angus is a Haskell programmer and he uses typeclasses and higher kinds everywhere."
Person A: "But no actual programmer obsesses over type-level programming."

Or alternatively:

Person A: "No Scotsman puts sugar on his porridge."
Person B: "But my uncle Angus is a Scotsman and he puts sugar on his porridge."
Person A: "But no true Scotsman puts sugar on his porridge."

On what basis do you conclude that type-level programming is somehow distinct from actual programming with any more validity than Person B concluding that Uncle Angus is not a true Scotsman? When you're about to make a comment that dismisses the fabulous work and contribution of thousands of engineers worldwide who write productive, useful code every day, you need to think very carefully about the implications of that message.

There is plenty of both commercial investment and top tier academic research in purely functional, type-level programming and it would be a brave move indeed to write it all off in a single paragraph.

I think you need to give more thought to what constitutes your view of "actual programming" and how the practitioners of it differ in their objectives from the practitioners of type-level programming. To create a facetious example, do "actual programmers" love their programs to crash with exceptions or run really slowly? Personally, I doubt it but whatever it is, once you've defined what constitutes that group, you might be able to figure out whether the two communities are capable of learning from one another.


I hope that, if nothing else, with this post I have provided some pause for thought about the nature of the arguments voiced against including type-classes, higher kinds, etc in F#. There may be totally valid reasons for not doing so but I'm not 100% sure you've got to the heart of a rational, reasoned, case against here. It may be that you just need to reflect on how that case against has been framed or it may also be that you need to consider more seriously whether that case actually exists.

Sorry, something went wrong.

@dabrahams

Pick your reaction

@dabrahams dabrahams commented 2 days ago

@dsyme wrote

I've translated Swift code using protocols to F#, and the use of Swift protocols was needless, and the resulting F# was much, much simpler (and much more like the original Python). It was very obvious to me that the person translating the original Python samples used Swift protocols to "show off" what so-called "value" Swift was bringing to the table. Well they failed - the use of protocols was simply obfuscating, not helpful.

I've seen protocols misused, but it doesn't sound like that's what was going on here. It's hard to tell for sure without seeing the code you're referencing, but if the Python code was making use of open polymorphism, then the use of protocols almost surely wasn't superfluous. Of course, there are other ways to get open polymorphism in Swift: through closures or traditional OO-style classes (both of which confer reference semantics), so protocols are never "necessary." But it's absolutely natural that a Swift program includes protocol declarations that don't appear in a corresponding (dynamically typed) Python program, just like most OO languages would require the declaration of base classes that would be unneeded in Python.

Ultimately this may come down to whether you believe static type systems are valuable. While your sense that the code was obfuscated by the use of protocols is certainly an important data point, the maintainability and performance benefits of static typing should not be overlooked either.

I'd like to make a small edit to your conclusion smile:

This happens all the time once extensive type-level programming facilities are available in a language - they immediately, routinely get mis-applied in ways that makes code harder to understand, excludes beginners and fails to convince those from the outside.

Swift protocols don't imply “type-level programming” any more than abstract base classes do. For what it's worth, protocol extensions can make even Generic Programming (per Stepanov) as simple as writing an ordinary method. I certainly have my complaints with details of how Swift defines protocols, but in my experience they have been excellent for Swift's standard library and approachable/understandable for its users.

Sorry, something went wrong.

@kspeakman

Pick your reaction

@kspeakman kspeakman commented 2 days ago

@TheInnerLight wrote:

(paraphrase) type level programming vs actual programming = no true scotsman fallacy

I took dsyme's comments to mean. Writing code in terms of types-level abstractions + knowing how those will solve my problem VS writing code to solve my problem. Not to discount the beauty of encoding the solution in math. But it adds a layer.

Sorry, something went wrong.

@dsyme

Pick your reaction

Collaborator

@dsyme dsyme commented 2 days ago

@TheInnerLight

First, I politely request that the use of the personal pronoun "you" (referring to a specific person) be minimised in replies to this discussion thread. That is not the mode we generally use in the F# community in technical discussions (obviously, there are exceptions). I've carefully gone over my reply to ensure this is followed. If the personalized mode of argument is used I or other admins may delete replies, and request reposting without making arguments in this way. I will clarify this in the repo README.

That said, I appreciate the points being made and offer some comments below.

On the plus side, more type-level programming has massive run-time performance upsides.

This should indeed be noted in the section at the end of my note, which describes positive uses of this kind of machinery for interop.

That said, source code generators for C# and Type Providers for F# can play similar roles (for F#, especially with #450, approved in principle). So if that's the outcome we're after then type-classes are not the mechanism we'd use to get it - indeed it argues for investing elsewhere.

Note that F# does embrace very extensive type-level programming via type providers. These suffer some of the above problems (e.g. they run at compile-time, and can cause performance and debugging problems). So some of what I've written can also be seen as a criticism of this existing part of F#, or of compile-time programming in general. However type providers do at least allow debug/performance-profiling/unit-testing paths for the type-level programming logic (since they are compiler plugins, where F# code runs in the toolchain), and avoid the type-level abstractions of category theory or abstract algebra.

Is [asking] for other language features so terrible that it deserves excluding useful techniques on that basis?

We love that people ask for stuff and we have 400+ open issues here for that.

That said, "the classic slippery slope" is indeed a basis on which to exclude otherwise useful language features in this repository. In the F# and C# design traditions we have long avoided features with have an obvious slippery slope - we aim to do them complete, as a whole, and resolve the design point to a sufficient coherent closure (though we often fail in that).

Surely it's positive that you have an active community that is talking about things

It depends entirely what they're talking about. For those looking for community forums or teams discussing category theory or abstract algebraic concepts, then there are other communities for that.

like the millionth iteration of the best form of dependency injection?

I'm glad this establishes the kind of base line we're talking about when it comes to useless community discussions :) We're not keen on dependency injection in the F# world and are blissfully free of those discussions. Great example.

Why [are] these conversations are less useful or productive than any of the other topics that come up in language communities

See the mention of dependency injection.

More usefully, there are have been plenty of highly productive community conversations over the years (e.g. the creation of Fable, or SAFE Stack or Bolero or FsCheck or WebSharper or ...). By the high standards of the F# community, arguing over hierarchical type classifications would be solidly in the unproductive camp.

...because it's an opportunity for people to learn things.

F# is not specifically a vehicle for educating people in new things, nor providing them with an opportuntity to learn new things. We could easily give our error messages in Latin, for example, or include links to category theory primers, or ask people to get to level 10 in Minecraft before their program compiles, or give links to half-relevant Knuth articles - all would be learning new things.

Instead, F# is a vehicle for productive programming. This might need a small amount of (re-)education, but (re-)eduction is absolutely not a goal in itself. If a team can productively use F# without any (re-)education that's fabulous!

Are the subtleties introduced by type classes notably worse than the subtleties introduced by any other language feature that has been successfully integrated with the language?

Yes, they are in the "pretty bad" category, and get worse the more type-level programming machinery is implicitly invoked. F# has some subtle features, it's true, but it goes without saying that we're not in the business of making that worse.

...setting the agenda...

To clarify: F# is not a research vehicle (it hasn't been since ~2014). "Setting the agenda" is not intrinsically a goal, and to the extent we do we have choices over what agenda we set. Sometimes this means not doing things. A good summary of the dimensions of expression-oriented programming which F# has advanced is in my talk "The Early History of F#" at the ML Workshop 2021. We will continue to advance along those dimensions that particularly align with overall productivity improvements and net-utility for programming teams.

Those looking for a language whose goal is specifically to be a research vehicle for cutting-edge ideas (e.g. advanced research in program verification, or touch-based-programming, or type-level programming, or functorial programming or similar) should look elsewhere.

but wouldn't necessarily need to be provided by FSharp.Core in any case

Unfortunately that would give rise to bifurcated communities, some depending on this foundational library, some not, some depending on other foundational libraries.

On what basis do you conclude that type-level programming is somehow distinct from actual programming

Please replace "actual programming" by "value-based programming" or "expression-level programming" or "programming with data and information" or "programs that run at runtime". In the context of .NET and F#, these are sufficiently well-defined and highly distinct from type-level programming ("programs or logic that runs at compile-time and compute with types") that I'm comfortable with the distinction.

As a joking aside - I suspect the strange porridge analogy doesn't apply, perhaps it's something to do with the milk being used? Or maybe he's really eating a burrito? :)

...write it all off in a single paragraph.

To be clear, this discussion thread is about the utility of adding type classes and its variations on type-level programming specifically in F#. Although it was put on twitter as an "attack" on type-level programming in general, and includes a speculative rant about psychology, it isn't intended as such, and the last paragraph clarifies that there are genuine dimensions of utility in this area - the rest is intended to give balance and rationale. As an example, Anders Hejlsberg has plenty of concern about how the type-level programming machinery of TypeScript is being over-deployed - but the foundational reason for why he's adding the features is genuine utility. He wouldn't be adding them if they weren't highly useful for interoperating with JavaScript. Other programming communities can do as they like.

Academic research should absolutely be doing type-level programming and whatever other things thay want. They should be teaching it too - including teaching its critiques. It is fascinating and examinable and trains in abstract thought. If anyone wants me to write an article or a supporting letter for a research grant on why type-level programming makes for a great part of the curriculum, I'd be more than happy to do that. It is easily prevalent enough in C++, Scala, Haskell, TypeScript and even typed-Python to make it valuable to know about - both its pros and cons.

Sorry, something went wrong.

@dsyme

Pick your reaction

Collaborator

@dsyme dsyme commented 2 days ago

@dabrahams

But it's absolutely natural that a Swift program includes protocol declarations that don't appear in a corresponding (dynamically typed) Python program, just like most OO languages would require the declaration of base classes that would be unneeded in Python.

Right, but the natural "best" F# program corresponding to a Python program generally doesn't use (explicit) protocols, nor does it necessarily use base classes. Instead, for the parts where these occur, it tends to use generic code with perhaps 1-3 function parameters to give adhoc meaning to the type parameters. These techniques are routine in F# and thoroughly well supported through HM type inference. SRTP may also be used for some things that Swift protocols may cover.

I'd like to make a small edit to your conclusion smile.. Swift protocols don't imply “type-level programming” any more than abstract base classes do.

Thanks, yes, that's a useful clarification. Swift protocols are certainly at the very, very light end of type-level classification and involve little computation.

Sorry, something went wrong.

@dabrahams

Pick your reaction

@dabrahams dabrahams commented 2 days ago

Right, but the natural "best" F# program corresponding to a Python program … tends to use generic code with perhaps 1-3 function parameters to give adhoc meaning to the type parameters. These techniques are routine in F# and thoroughly well supported through HM type inference. SRTP may also be used for some things that Swift protocols may cover.

I think you're well beyond my knowledge of F# at this point, but it sounds like this means the “best” F# program is prone to diagnostic backtraces like those that result from unbounded type inference in Haskell, or for that matter, template instantiation in C++. When everything in the program type-checks, the type annotations might seem like noise, but during development and maintenance they are essential, produce a better experience for library users, and ultimately reduce the amount of documentation required. Maybe this just proves F# isn't a language for me, but IMO programming is barely tenable when a type error in a caller can produce a diagnostic in its callee.

Sorry, something went wrong.

@drvink

Pick your reaction

@drvink drvink commented 2 days ago

it sounds like this means the “best” F# program is prone to diagnostic backtraces like those that result from unbounded type inference in Haskell, or for that matter, template instantiation in C++

I think Don means exactly the opposite of that: the "best" F# program doesn't make heavy use of SRTP and therefore such diagnostic noise is kept to a minimum (because typechecking errors are precise under these circumstances; otherwise you indeed have a similar problem to the one with C++-style simulation of typeclasses via templates).

Sorry, something went wrong.

@dsyme

Pick your reaction

Collaborator

@dsyme dsyme commented 2 days ago

Maybe this just proves F# isn't a language for me, but IMO programming is barely tenable when a type error in a caller can produce a diagnostic in its callee.

I understand, but I believe this is ok in a typical F# IDE, doing whole-file type inference and error reporting on-the-fly, with appropriate type information displayed in tooltips and codelens for partially complete programs. At least, I think that would be enough to be comfortable with the kind of type reconstruction and generalization that F# does.

Sorry, something went wrong.

@gusty

Pick your reaction

@gusty gusty commented 2 days ago

Just to clarify: I'm still against adding type classes without HKs.

Category theory (focused in computer science) definitely matters as it provides a "framework reference" to make ad-hoc overloading less ad-hoc.

This has currently an impact in (mostly library) F# programming and I've seen many times people hitting a wall after adding overloads in a disorganized way, which doesn't follow any common abstraction, and getting logically type inference confused in client code, which forces at some point a breaking change in their design, but it really takes long to realize that, sometimes it never happens.

Moreover, those compiler errors are actually way more cryptic than say Haskell-like errors like "an instance for Monad is missing in type X". But even if/when those technical problems are sorted out, the resulting API is so complicated to use that we have to resort to docs or source code to know which kind of black magic the overload resolution is providing. On the same track, I'm convinced type inference would speed up if validating against type classes as opposed to a big chunk of unrelated overloads.

I am not talking about advanced CT abstractions but simple day-to-day use, specially in library design, like Monads, Applicatives , Functors and Traversables, same ones that are explained nicely without requiring any degree in Maths in the acclaimed F# for fun and profit website, with real world examples.

I think no one can deny those abstractions are present today in F# code and probably key language features like CEs weren't a reality today if those abstractions were always completely "banned" to use in programming languages.

Now, using say Monads and not having a way to express them in the type system, complicates understanding them, and kind of relegate it use to language designers or library designers, a situation which actually promotes this:

lead to communities where the most empowered programmers are those with deep expertise in certain kinds of highly abstract mathematics

I mean, nowadays developers who know other languages where HKs are a reality, are in a better position than the rest to create (or port to F#) libraries (I think I don't need to give examples of such popular F# libraries). So this is effectively causing such division. Reflecting the real type it represents in the type system would help consumers to understand the abstractions, i.e. it would "socialize" the abstraction, instead of keeping it as a secret sauce for the library (or language) designer.

It's important to clarify that even in that hypothetical case, knowing CT won't be a requirement to use such library.

Now let's forget about CT, higher kinds are still useful, they would allow for instance to better express functions dealing with unit of measures, also it would allow better expressing constraints to our domain model, when used in a proper way. Could it be abused? of course, like any other existing feature.

My personal experience while learning generics was from time to time getting surprised when finding the HK limitation as I was not aware of Kinds at all, so it was more complicated to deal with that limitation and the compiler error was cryptic to me. I was wondering "if this allows generic types, why can they only be defined in certain cases".

So, I think Higher Kinds are a natural evolution and the same way generics were criticsized 20 years ago, it will be seen as a natural way of expressing types for new generations of developers. And by the way, this whole discussion reminds me in general the way FP was regarded 25 years ago, like an academic thing only for people interested in the math behind computer programs.

Sorry, something went wrong.

@TheInnerLight

Pick your reaction

@TheInnerLight TheInnerLight commented yesterday

...I appreciate the points being made and offer some comments below.

Likewise, I appreciate the time taken to reply.

Note that F# does embrace very extensive type-level programming via type providers. These suffer some of the above problems (e.g. they run at compile-time, and can cause performance and debugging problems). So what I've written can also be seen as a criticism of this existing part of F#, or of compile-time programming in general.

Isn't the type of type level programming that is discussed above very far beyond, in terms of complexity, the use of type-classes as per the topic of this thread?

F# type providers are a very nice piece of technology and probably one of the stand-out features of the language but they fall more into the same category of features as meta-programming features as Template Haskell and Scala macros. It's hard to imagine even the most ambitious Scala or Haskell type astronaut recommending reaching for them too quickly.

They are a fine solution if one wishes to automatically generate types from a database or huge schema file but come with dramatically more scope to shoot oneself in the foot than a lawful typeclass representing something like a Semigroup.

These are obviously at opposite ends of the spectrum but is it desirable to push people seeking type-level features to the most complicated extreme?

However type providers do at least allow debug/performance-profiling/unit-testing paths for the type-level programming logic (since they are compiler plugins, where F# code runs in the toolchain), and avoid the type-level abstractions of category theory or abstract algebra.

It's good that those features are available if needed but debugging is not at all desirable. If one has reached the point of firing up a debugger in order to understand their program flow, it's probably crossed a serious complexity barrier and the feedback loop is going to be slow and less productive. Unit tests are better, the feedback loop is faster and, especially with property tests, it's possible to cover things pretty exhaustively if you're extremely careful.

Neither of them are remotely as fast and ergonomic as type errors directly fed back by the compiler.

like the millionth iteration of the best form of dependency injection?

I'm glad this establishes the kind of base line we're talking about when it comes to useless community discussions :) We're not keen on dependency injection in the F# world and are blissfully free of those discussions. Great example.

I am pleased that F# has transcended such discussions, everyone in the community is totally clear on how to architect programs of arbitrary size, and the F# community is "blissfully free" of those discussions.

It is, in fact, so "blissfully free" of them, that a quick google yields an inexhaustive list of some massive ones, some of which amount to practically a small book about the topic.

Even if we don't actually talk about dependency injection explictly and instead talk about free monads, effect systems, tagless final and ReaderT patterns, we're still fundamentally talking about how to architect programs achieving the best compromise of abstraction, readability, correctness, maintainability and testability. The answer of how best to do that certainly hasn't been answered. Maybe it will never be answered as the techniques we use just continue to slowly evolve.

Even if we've had these conversations a million times and, mostly, little progress is made, the only thing worse than people talking about them is people not talking about them, because that means they've given up on solving the problem.

More usefully, there are have been plenty of highly productive community conversations over the years (e.g. the creation of Fable, or SAFE Stack or Bolero or FsCheck or WebSharper or ...). By the high standards of the F# community, arguing over hierarchical type classifications would be solidly in the unproductive camp.

Meanwhile, were the Scala community so absorbed by arguing over hierarchical type classifications that they were unable to produce Http4s, Cats, Circe, Doobie, FS2 and Scalajs? Or did it prevent Haskellers producing Yesod, Hasql, Reflex, Pandoc, XMonad or IHP?

It's not clear if this comment is supposed to imply otherwise but I think it's pretty safe to conclude that the existence of hierarchical type classifications is orthogonal to community achievement. I trust that is not in dispute.

Adding any new feature results in those who know how to use that feature having an advantage over those who don't yet know how to use it. This is great because it's an opportunity for people to learn things. If we didn't like learning things, we'd never have learnt F#.

F# is not specifically a vehicle for educating people in new things, nor providing them with an opportuntity to learn new things. We could easily give our error messages in Latin, for example, or include links to category theory primers, or ask people to get to level 10 in Minecraft before their program compiles, or give links to half-relevant Knuth articles - all would be learning new things.

My whole quote has been reintroduced here because the original point has actually not been considered. Let me reiterate that every new language feature is something that most people don't currently know how to use.

The fact that some people do not currently know how to use new language features has not, seemingly, been a reason to freeze all language development on F# so far. That being the case, I'm struggling to understand how it is problematic to introduce well studied, proven, concepts from both other languages and academia simply because people who have spent years of their lives devoted to studying those things know more about them than people who have not. Surely it's reassuring to know that we can introduce proven concepts into the language, rather than something half baked on the back of an envelope?

I'm also struggling to see how introducing well proven concepts from computer science to a programming language bears any relation to including Minecraft or Latin in the compiler (although maybe Microsoft have done weirder things in the past laughing).

Instead, F# is a vehicle for productive programming. This might need a small amount of (re-)education, but (re-)eduction is absolutely not a goal in itself. If a team can productively use F# without any (re-)education that's fabulous!

That's great. I am pleased to hear that F# is for productive programming. Typeclasses are a language feature that can be highly productive. Sounds like F# and typeclasses could be a match made in heaven :)

Those looking for a language whose goal is specifically to be a research vehicle for cutting-edge ideas (e.g. advanced research in program verification, or touch-based-programming, or type-level programming, or functorial programming or similar) should look elsewhere.

What if someone is looking for a strongly typed, functional programming language that embraces type-level programming for industry productivity? It would be concerning, in my view, if F# concluded it was not for such people.

I have tremendous respect for all the academics who contribute to the world of functional programming research and there's lots of exciting things going on that field that are set to contribute massively to the field of software engineering in the future. Ultimately though, my interest is primarily in building really scalable, maintainable, reliable web services. There are well proven, well understood, type level programming techniques that help to do that and if people interested in that sort of thing are encouraged to go elsewhere, they will, and they will take their opportunities for commercial functional programming with them.

Unfortunately that would give rise to bifurcated communities, some depending on this foundational library, some not, some depending on other foundational libraries.

Sounds like a community with a diverse set of opinions and ways of doing things within, with different parts that could learn a lot from each other, can imagine that would be awful :)

Please replace "actual programming" by "value-based programming" or "expression-level programming" or "programming with data and information" or "programs that run at runtime". In the context of .NET and F#, these are sufficiently well-defined and highly distinct from type-level programming ("programs or logic that runs at compile-time and compute with types") that I'm comfortable with the distinction.

So, it is indeed about programs that annoyingly fail at runtime then?

I work a lot with Json and Avro formats, it's useful to be able to serialise data into these formats, it's a simple, value-based, data-based programming problem. I can construct a program in F# and in Haskell or Scala to do that. In fact, the only difference between the F# and Scala/Haskell versions of this code is that the F# has extra failure modes because I can't guarantee that I don't try to encode something absurd.

It's actually worse than the F# version just having an additional failure mode though. Since the serialisation is based on reflection, I have a massive security vulnerability since a small typo can lead to me readily exposing the wrong data via my web API. In both Haskell and Scala I have to explicitly mark a type as a encodable to JSON for it to be capable of being exposed to the outside world, in F#, reflection will magically figure out how to do that, no matter how horrible the consequences of that.

To be clear, this discussion thread is about the utility of adding type classes and its variations on type-level programming specifically in F#. Although it was put on twitter as an "attack" on type-level programming in general, and includes a speculative rant about psychology, it isn't intended as such...

I'd like to conclude by adding that should a contributor here not wish their comments to be taken as attacks on the type-level programming community, they should avoid making statements that dismiss a very large and diverse community to some special category of not "actual" programming regardless of their intent when doing so and should instead focus on critique of the specific type-level programming techniques in question.

Sorry, something went wrong.

@fc1943s

Pick your reaction

@fc1943s fc1943s commented yesterday

can you take a look at spiral lang and see if it does fit the needs of F# + type stuff? also how the serialization work https://github.com/mrakgr/The-Spiral-Language

Sorry, something went wrong.

@cartermp

Pick your reaction

Member

@cartermp cartermp commented yesterday

What if someone is looking for a strongly typed, functional programming language that embraces type-level programming for industry productivity? It would be concerning, in my view, if F# concluded it was not for such people.

This is the core of the matter, and the answer is that F# is not for such people. For that, there are some options (Haskell, PureScript, perhaps Scala depending on what you're into).

Just to clarify this point:

  • Libraries like FSharpPlus are fine and it's good that they can be authored and used and enjoyed by people
  • Bugs that arise from authoring libraries like FSharpPlus tend to get fixed over time
  • There's no code that says you're a bad person when you get super creative with SRTP
  • SRTP will continue to see improvements and thus the current avenue for this style of programming is improving

But if you're looking for a home as a "I wish to embrace type-level programming in my language" kind of programmer, you should look elsewhere. You can be plenty productive in F# without hierarchies of this nature, but if that's not what you want, then you'll be a lot more satisfied with a different language.

Sorry, something went wrong.

@DalekBaldwin

Pick your reaction

@DalekBaldwin DalekBaldwin commented yesterday

Some purely mundane and non-philosophical points on this matter:

Type providers are tricky to understand and implement, but once you have one, client code can be written against it without having to understand any new type-system features. You just call normal methods and properties on what is for all practical purposes another normal .NET type, and the design decisions in TPs and the TP SDK don't ripple throughout the whole ecosystem. A new F# feature might be able to replace something you previously could have implemented with clever type provider magic, but TPs are mostly orthogonal to the design of the F# type system itself.

Haskell's status as a research language affects how easily even "practical" new features can be incorporated into it. Lots of features that are de-facto standard and stable at this point still require opt-in language pragmas (often a combination of pragmas). In .NET, you basically just have <LangVersion>preview</LangVersion> and <EnablePreviewFeatures>true</EnablePreviewFeatures>, with a purely linear progression.

Also, any new F# type-level features that can't be encoded at the level of the IL itself raise a whole host of interop issues. If I write a HKT/typeclass-based library in F#, can I use it to create a C# application? In an F# application, can I use a framework written in C# and instantiate it with things I've defined using the latest F# feature? (Functions with SRTPs, which form the basis of most demonstrations of advanced type-level programming in F# floating around the web, can't be called from C#).

So arguments about new type-level programming features are more likely to derail the development of F# than Haskell. Even though there's been lots of cross-pollination of ideas between F# and the broader .NET ecosystem, to steer the ship of F# in some directions, you would need to steer the whole ocean.

Sorry, something went wrong.

@dsyme

Pick your reaction

Collaborator

@dsyme dsyme commented yesterday

Responding to a few parts:

@cartermp

This is the core of the matter, and the answer is that F# is not for such people.

@cartermp - I believe type providers should be included in your list of available variations on type-level programming techniques

@TheInnerLight

Isn't [type level programming via type providers] very far beyond, in terms of complexity, the use of type-classes as per the topic of this thread?

In practice no, not in terms of actual experienced complexity, nor in terms of the skillset required. When authoring type providers, the type-level programs are expressed using term-level programming with all the utility of the expression language of F# itself (that is, type providers are macro-like compiler addins authored with the F# expression language, manipulating and producing reflection objects - System.Type and quotations - at compile-time, and programming with these reflection objects is already known to most F# programmers). This means relatively good programming, debugging, profiling, logging, diagnostic reporting etc. is available for type-provider authors (though it could be improved, and there are many gotchas).

In contrast, if the logic being implemented by typical type providers (e.g. validating SQL, or parsing HTML/JSON/CSV/XML) were instead encoded in type-level programming with, say, type classes, implicits, constraints, type-level conditionals, type-level loops, type-level web-requests etc. then it would be utterly intractable, entirely baroque and impossible to debug. At least it's beyond my capabilities to make F# into a language where this is simple enough. One can write JSON parsers using TypeScript type-level constraint logic but I'm not ever going down that path with F#, I'm really not. Especially given that most practical industrial uses are covered (or potentially covered) by type providers (F#) and source code generators (C#).

If one has reached the point of firing up a debugger in order to understand their program flow, it's probably crossed a serious complexity barrier and the feedback loop is going to be slow and less productive.

As mentioned by @DalekBaldwin authoring new type providers may require debugging and testing. However the usage model of a completed, tested type provider is based around the provided nominal types and their methods/properties, and is generally very simple, and does not need debugging or testing pathways for the type-level logic.

F# Myriad is another approach worth mentioning. Source generators for C# are also an important comparison point, as is T4 template meta-programming. Again the result in each case is nominal types, and the delivered usage model of each mechanism simple. C# and F# analyzers are also relevant, again implemented using compiler-addins operating on reflection-like views of programs (from Roslyn and FSharp.Compiler.Service), including partially completed programs.

What if someone is looking for a strongly typed, functional programming language that embraces type-level programming for industry productivity?

If that is the specific aim, I would point them to the specific kind of reflective type-level programming done using F# type providers, or C# code generators, or F#/C# analyzers. all of which have many proven practical industry uses, sometimes overlapping. I would also apologise that #450 has not been landed as yet (it is certainly useful for type providers to take types defined in the same assembly as inputs).

...they should avoid making statements that diminish its achievements...

If any critique of type-level programming ("diminishing its achievements") is an attack, then type-level programming has become holy - a thing that may not diminished. The problem is, that itself could be added to the list of negatives....

Sorry, something went wrong.

@TheInnerLight

Pick your reaction

@TheInnerLight TheInnerLight commented 20 hours ago

In contrast, if the logic being implemented by typical type providers (e.g. validating SQL, or parsing HTML/JSON/CSV/XML)...

I would tend to agree that this is a good use case for something like type providers, indeed doobie in Scala which I previously mentioned uses macros to do this. So far we're on firm footing in terms of using equivalent features to do like with like.

The subtlety is that doobie introduces various type classes to permit values of particular types to be successfully rolled into or extracted as results from SQL queries. Doobie provides instances in the base library for common types and then various extensions (some of which could even be authored by other contributors) provide additional, more specific instances. An example being support for postgis types.

This is a pretty good story for everyone involved because the macro/type provider author wouldn't need to provide a mapping for every use case, they provide the structural mapping to SQL and typeclasses provide the specific ability to work with individual types (and if we make mistakes and use incorrect types, we still get nice compiler errors) so a nice example of the two things working together.

... were instead encoded in type-level programming with, say, type classes, implicits, constraints, type-level conditionals, type-level loops, type-level web-requests etc. then it would be utterly intractable, entirely baroque and impossible to debug. At least it's beyond my capabilities to make F# into a language where this is simple enough. One can write JSON parsers using TypeScript type-level constraint logic but I'm not ever going down that path with F#, I'm really not. Especially given that most practical industrial uses are covered (or potentially covered) by type providers (F#) and source code generators (C#).

This is trickier, I agree. If we get into using Haskell Generics, Template Haskell or Shapeless in Scala to generate JSON instances for example, we've potentially re-introduced some of the same problems as I previously called out with Reflection. It saves a lot of time but we're at the mercy of the algorithm used to structurally generate that type mapping.

It's still better than reflection, it still avoids compiler errors, is fast at runtime, forces the author to mark types they want to be explicitly decodable or encodable but it's harder to customise specific behaviour we might be looking for on a per field basis.

I think both of these approaches are absolutely valid but, particularly for production systems, I'd be much more comfortable with hand written ToJSON instances than automagically derived ones even if it results in some extra boilerplate. Maybe that's a prejudice on my part though. Someone else who has written a lot of Scala might be able to provide a stronger case in favour of structural deriving if those involved are eager to seek that out in the name of diversity of opinions.

If that is the specific aim, I would point them to the specific kind of reflective type-level programming done using F# type providers, or C# code generators, or F#/C# analyzers. all of which have many proven practical industry uses, sometimes overlapping. I would also apologise that #450 has not been landed as yet (it is certainly useful for type providers to take types defined in the same assembly as inputs).

I think this is a good start. As I pointed out above, this is somewhat orthogonal to the question of type classes because one can use both of these approaches together to produce better results than with one of the approaches alone.

If any critique of type-level programming ("diminishing its achievements") is an attack, then type-level programming has become holy - a thing that may not diminished. The problem is, that itself could be added to the list of negatives....

I'd like to apologise for the lack of clarity there and the somewhat clumsy wording of that particular clause of my final comment which has caused some confusion and I will modify it to clarify the intent. I trust that we're all capable of drawing a distinction between specific critique of elements of type-level programming which, I agree, are absolutely called for in the context of this thread and broad community bashing which is absolutely not.

Sorry, something went wrong.

@gusty

Pick your reaction

Copy link
Reference in new issue

Reference in new issue

fslang-suggestions
Repositories

@gusty gusty commented 11 hours ago

I agree with @TheInnerLight in that both features seems to be orthogonal.

I think if we had both normal HKs and type providers (with the pending features above mentioned) there won't be any need to go down the road of complicated type level programming, since any advanced type level technique beyond normal HKs would be easier to achieve with type providers. That would effectively stop that slippery slope.

Nowadays we don't have either: we have generics but limited to first order kinds, type providers that can't generate types from other types, oh and SRTPs that ignore extensions. That's probably one of the reasons this suggestion have so many votes, since as library developers we can't go too far and I'm under the impression that many devs have the (wrong) feeling that typeclasses alone (without HKs and other stuff) would solve lot of these limitations.

Why don't we see so much progress? This is probably due to lack of resources / time to implement the above mentioned features, but HKs is a bit special as it was stated before that it won't happen until the CLR team implement them, but now I wonder if this statement means that even when the CLR team implement them and while C# embraces Higher Kinds, F# will forbid them. That would be really unfortunate IMHO.

Note that if we have all above features (specially HKs) the lack of typeclasses won't be a big issue as we'll be able to use other similar techniques like SRTPs (but with real HK type constraints) or possibly the proposed static interfaces.

@DanielBMarkham
Select a reply ctrl .
Remember, contributions to this repository should follow its code of conduct.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet